Hashing packet contents to determine a processor

ABSTRACT

The disclosure includes a description of an apparatus having circuitry to determine a first hash value for a first packet tuple of a first packet traveling in a first direction of a duplex connection and determine a processor for the first packet from a set of multiple processors based, at least in part, on the first hash value. The apparatus includes circuitry to determine a second hash value for a second packet tuple of a second packet traveling in a second direction of the duplex connection and determine the same processor for the second packet from the set of multiple processors based, at least in part, on the second hash value.

BACKGROUND

Networks enable computers and other devices to communicate. For example,networks can carry data representing video, audio, e-mail, and so forth.Typically, data sent across a network is divided into smaller messagesknown as packets. By analogy, a packet is much like an envelope you dropin a mailbox. A packet typically includes “payload” and a “header”. Thepacket's “payload” is analogous to the letter inside the envelope. Thepacket's “header” is much like the information written on the envelopeitself. The header can include information to help network deviceshandle the packet appropriately. For example, the header can include anaddress that identifies the packet's destination.

A series of related packets can form a connection. A connection is oftenidentified by a combination of different portions of a packet known as atuple. For example, a tuple is commonly formed by a combination ofsource and destination information of a packet header.

A variety of networking protocols maintain state information for aconnection. For example, the Transmission Control Protocol (TCP) storesstate data for a connection in a Transmission Control Block (TCB). A TCBincludes state data such as the last received byte, the lastsuccessfully transmitted byte, and so forth. Typically, connection statedata is accessed and, potentially, updated for each packet in aconnection. In a multi-processor system, this can create contentionissues between processors handling packets for the same connection. Thatis, for example, different processors handling data for the sameconnection may each attempt to access a connection's state data at thesame time, creating requirements for data locking and introducing delayas the processors wait for access to the connection state data.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of a system that determines a processor for a packetusing a symmetric hash.

FIG. 2 is a diagram illustrating a symmetric hash.

FIG. 3 is a diagram illustrating a network interface controller.

DETAILED DESCRIPTION

In a multi-processor system, processors may vie for access to the sameconnection state information. Contention between the processors,however, can be reduced by mapping respective connections to therespective processors. For example, a network interface controller (NIC)may perform a hash on a tuple of a received packet and use the hash todetermine a processor to handle a given packet. Directing packets havingthe same tuple to the same processor can help pin down state informationto the same processor. This can enable the processor to retain the statedata for a connection in local processor memory (e.g., cache memory) andreduce contention between processors trying to access the sameconnection state data.

Intermediate nodes in a network such as a security gateway, firewall,switch, or router may handle data traveling in both directions of aduplex (i.e., bi-directional) connection. For example, FIG. 1 depicts amulti-processor (e.g., multi-core) 102 a-102 n host 100 handling packetsof a duplex connection between nodes “A” and “B”. The processors 102a-102 n may be integrated on a single die and/or may be included withinthe same integrated circuit package. The processors 102 a-102 n each mayfeature programmable logic such as an instruction decoder, arithmeticlogic unit, and so forth. As shown, the processors 102 a-102 n may becoupled to and commonly service packets received by NICs 104 a, 104 b.Processors 102 a-102 n may communicate with the NICs 104 a, 104 b via achipset, interconnect, or other inter-communication circuitry.

In the example shown in FIG. 1, packets (e.g., 110 a) traveling fromnode A to node B have a source of “A” and a destination of “B” whilepackets (e.g., 110 b) traveling from node B to node A have a source of“B” and a destination of “A”. As shown, the host 100 receives packet 110a at NIC 104 a and packet 110 b at NIC 104 b. Both NICs 104 a, 104 b mapreceived packets to a selected processor 102 a-102 n.

A NIC 104 a, 104 b may use an asymmetric hash that yields a differenthash value for a packet in a connection depending on the direction thepacket travels (e.g., a hash where hash(Source A, Destination B) doesnot equal hash(Source B, Destination A)). In this case, the NICs 104 a,104 b may map packets belonging to the same connection to differentprocessors 102 a-102 n due to the different hash values derived forpackets traveling different directions in the same connection. This mayundermine a goal of reducing contention between processors 102 a-102 nfor connection state data. That is, if packet 110 a is mapped toprocessor 102 a and packet 110 b is mapped to processor 102 n, thenprocessors 102 a and 102 n may both vie for access to the connectionstate data for the connection between nodes A and B.

As shown in FIG. 1, NICs 104 a, 104 b may instead use a processorselection operation that features a symmetric hash that yields the samehash value for a packet in a connection regardless of the direction thepacket travels (e.g., a hash where hash(Source A, Destination B)=hash(Source B, Destination A)). Such a hash may map packets belonging to thesame duplex connection to the same processor, processor 102 a in thisexample. In other words, due to generation of the same hash value forpackets traveling in both directions of a connection despite packet datavariations (e.g., different source and destination information), packetsbelonging to the same connection can be mapped to the same processor 102a. This can reduce cache thrash and contention between processors 102a-102 n for connection state data.

FIG. 2 depicts a sample technique to generate a symmetric hash. Asshown, circuitry 200 operates on different orders of the same bits ofpacket data. For example, in the illustration, asymmetric hash circuitry202 a and 202 b operates on switched orders of source/destination datafor a TCP/IP tuple. That is, hash 202 a operates on a tuple formed by:

-   -   {source IP, destination IP, source TCP port, destination TCP        port}        while hash 202 b operates on a tuple formed by:    -   {destination IP, source IP, destination TCP port, source TCP        port}.

The output of circuitry 202 a and 202 b is then combined. For example,the output of hash circuitry 202 a and 202 b may undergo a combinationoperation 204 such as a logical AND and/or XOR. Thus, in this sampleimplementation, the circuitry 200 can form a symmetric hash fromasymmetric hash engines/functions 202 a, 202 b. This can enable thecircuitry 200 to use commonly implemented asymmetric hash engines (e.g.,Toeplitz hash engines) to generate a symmetric hash, lowering the designcost of the circuitry 200.

While FIG. 2 depicts a parallel implementation of the circuitry, otherimplementations may vary. For example, in a serial implementation, thedifferent sets of bits may be fed to the same hash circuitry in turn. Awide variety of other techniques may be used to generate a symmetrichash. For example, protocol data may be sorted before a hash operation.For instance, a symmetric hash can be produced by circuitry that ordersIP address within a tuple by magnitude and TCP ports within a tuple bymagnitude and feeds the single ordered set of tuple data to a singlehashing circuit. Thus, in FIG. 1, both packets 110 a and 110 b wouldyield the same ordered set of data to be hashed, produce the same hashvalue, and may be mapped to the same processor 102 a.

Once determined, a symmetric hash value may then be used to determine aprocessor mapped to a packet's connection. For example, a mask may beapplied to the symmetric hash value and may be used as a lookup valueinto an indirection table that associates the masked hash values toprocessor numbers. The resulting processor number from the indirectiontable may be adjusted, for example, by incrementing by a basecore/processor number. After a processor is determined for a packet, thepacket may be queued, for example, in a processor specific queue. Aninterrupt may then be generated to the processor. Potentially, interruptmoderation may be used to reduce the number of interrupts signaled.

While FIG. 2 depicted a tuple of the source and destination IP addressesand source and destination TCP ports, other tuples may be formed. Forexample, a tuple may consist solely of the IP source and destinationaddresses. Alternately, or in addition, a tuple may include informationfrom other header fields, headers in lower layers (e.g., Ethernet) orhigher layers in a protocol stack (e.g., HTTP (Hypertext TransferProtocol) data or eXtensible Markup Language (XML) data), a packet'spayload, and/or portions thereof. Further, while the above genericallyreferred to Internet Protocol datagrams, this term encompasses both IPv4(Internet Protocol version 4) and IPv6 (Internet Protocol version 6)datagrams. Similarly, while the above described IP datagramsencapsulating TCP segments, other layer 3 or layer 4 protocols (e.g.,User Datagram Protocol [UDP]) in OSI (Open Systems Interconnection)terminology may similarly use the techniques described above. Finally, asymmetric hash may also operate on data not found in a packet (e.g.,identification of the NIC receiving a packet).

FIG. 3 depicts a sample NIC 300 implementing a symmetric hash. As shown,the NIC 300 includes a PHY 302 (physical layer devices) (e.g., wired orwireless PHYs) and a MAC (media access control). The NIC 300 may alsofeature a DMA (Direct Memory Access) engine to transfer packet data tohost memory (not shown) or directly to a host processor for example viaa chipset, interconnect, or other communication medium. In the sampleshown, the NIC 300 includes symmetric hash circuitry 304 for use indetermining a processor 102 a-102 n to handle a packet.

A NIC, such as NIC 300, can be configured to operate in either symmetricor asymmetric hash mode. For example, a NIC may be configured to use aparticular hash function (e.g., Toeplitz) and/or whether to generate asymmetric or asymmetric hash. For instance, this configuration may beperformed via a network driver executed by a processor. For example, thenetwork driver may specify an object identifier with the desiredconfiguration values/selection of asymmetric or symmetric hash.

While FIGS. 1-3 depict sample implementations and sample environments,many other implementations are possible. For example, the system of FIG.1 may feature a single NIC or more than two NICs that determine asymmetric hash. Further, the symmetric hash circuitry need not belocated in a NIC, but may instead may be located elsewhere in the host,such as in a chipset, processor 102 a-102 n circuitry, or instructionsexecuted by a processor 102 a-102 n. Additionally, while the abovedescribed an intermediate node in a network, the techniques describedabove may also be used in a terminal network node (e.g., a server).Further, while described in conjunction with bi-direcitonal connections,the techniques described above may also work with multi-casting orn-directional connections.

The term packet as used herein encompasses protocol data units (PDUs)for a wide variety of network protocols featuring a header and payload.A packet may be an encapsulated or encapsulating packet. Further, agiven tuple may feature data from zero or more encapsulated packetheaders and may or may not feature data from an encapsulating packetheader.

The techniques described above may be implemented in a variety ofsoftware and/or hardware architectures. The term circuitry as usedherein includes hardwired circuitry, digital circuitry, analogcircuitry, programmable circuitry, and so forth. The programmablecircuitry may operate on computer programs.

Other embodiments are within the scope of the following claims.

1-15. (canceled)
 16. A method, comprising: for a first packet receivedat a network interface of a system comprising multiple processors:ordering the Internet Protocol source address and the Internet Protocoldestination address of the first packet by magnitude and ordering thesource port and destination port of the first packet by magnitude;performing a hash based, at least in part, on the ordering of theInternet Protocol source address and the Internet Protocol destinationaddress of the first packet by magnitude and the ordering of the sourceport and destination port of the first packet by magnitude; anddetermining a processor from the multiple processors based on theperformed hash.
 17. The method of claim 16, further comprising: for asecond packet to be transmitted via the network interface to a remotedestination: ordering the Internet Protocol source address and theInternet Protocol destination address of the second packet by magnitudeand ordering the source port and destination port of the second packetby magnitude; performing a hash based, at least in part, on the orderingof the Internet Protocol source address and the Internet Protocoldestination address of the second packet by magnitude and the orderingof the source port and destination port of the second packet bymagnitude.
 18. The method of claim 16, wherein the determining theprocessor comprises using the performed hash to perform a lookupassociating hash values with indications of processors.
 19. The methodof claim 16, wherein the hash comprises a Toeplitz hash.
 20. The methodof claim 16, wherein the determining the processor comprises selecting aqueue associated with the processor.
 21. A computer program, disposed ona non-transitory computer readable medium, comprising instructions tocause circuitry to: for a received packet: order the Internet Protocolsource address and the Internet Protocol destination address of thefirst received packet by magnitude and order the source port anddestination port of the first received packet by magnitude; performing ahash, based at least in part, on the ordering of the Internet Protocolsource address and the Internet Protocol destination address of thereceived packet by magnitude and the ordering of the source port anddestination port of the first received packet by magnitude; anddetermine a processor from a set of multiple processors based on theperformed hash.
 22. The computer program of claim 21, further comprisinginstructions for causing circuitry to: for a second packet to betransmitted via the network interface to a remote destination: order theInternet Protocol source address and the Internet Protocol destinationaddress of the second packet by magnitude and order the source port anddestination port of the second packet by magnitude; perform a hash,based at least in part, on the ordering of the Internet Protocol sourceaddress and the Internet Protocol destination address of the secondpacket by magnitude and the order of the source port and destinationport of the second packet by magnitude.
 23. The computer program ofclaim 21, wherein the instructions to determine the processor compriseinstructions to use the performed hash to perform a lookup associatinghash values with indications of processors.
 24. The computer program ofclaim 21, wherein the hash comprises a Toeplitz hash.
 25. The computerprogram of claim 21, wherein the determining the processor comprisesselecting a queue associated with the processor.
 26. A system,comprising multiple processors; at least one network interfacecontroller coupled to the multiple processors; and circuitry to: for areceived packet: order the Internet Protocol source address and theInternet Protocol destination address of the received packet bymagnitude and order the source port and destination port of the receivepacket by magnitude; perform a hash based, at least in part, on theordering of the Internet Protocol source address and the InternetProtocol destination address of the received packet by magnitude and onthe ordering of the source port and destination port of the receivedpacket by magnitude; and determine a processor from the multipleprocessors based on the performed hash.
 27. The system of claim 26,wherein the circuitry comprises circuitry to: for a second packet to betransmitted via the network interface to a remote destination: order theInternet Protocol source address and the Internet Protocol destinationaddress of the second packet by magnitude and order the source port anddestination port of the second packet by magnitude; and perform a hash,based at least in part, on the ordering of the Internet Protocol sourceaddress and the Internet Protocol destination address of the secondpacket by magnitude and the ordering of the source port and destinationport by magnitude.
 28. The system of claim 26, wherein the circuitry todetermine the processor comprises circuitry to use the performed hash toperform a lookup associating hash values with indications of processors.29. The system of claim 26, wherein the hash comprises a Toeplitz hash.30. The system of claim 26, wherein the circuitry to determine theprocessor comprises circuitry to select a queue associated with theprocessor.
 31. The system of claim 26, wherein the circuitry comprisescircuitry programmed by instructions disposed on a non-transitorycomputer readable medium.